(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 

International Bureau 

(43) International Publication Date 
3 January 2003 (03.01.2003) 




PCT 



llllll II INI II III III llll III II III 

(10) International Publication Number 

WO 03/001475 Al 



(51) International Patent Classification 7 : G08G 5/00, 

G09G 5/08, 5/39, G08C 21/00, G06T 1/60 

(21) International Application Number: PCTYUS02/19364 

(22) International Filing Date: 17 June 2002 (17.06.2002) 

(25) Filing Language: English 

(26) Publication Language: English 



(30) Priority Data: 

09/886,401 
09/886,399 



20 June 2001 (20.06.2001) US 
20 June 2001 (20.06.2001) US 



(71) Applicant (for all designated States except US): 
LEAPFROG ENTERPRISES, INC. [US/US]; 6401 
Hollis Street, Suite 150, Emeryville, CA 94608-1070 
(US). 

(72) Inventors; and 

(75) Inventors/Applicants (for US only): WOOD, Michael, 



C. [US/US]; 27 La Nona, Orinda, CA 94563 (US). JE- 
DRZEJEK, Tracey, Hope [US/US]; 3364 La Mesa Drive 
#12, San Carlos, CA 94070 (US). FREEMAN, Richard, 
Glen [US/US]; 1307 Crcstvicw Drive, San Carlos, CA 
94070 (US). FLOWERS, Mark [US/US]; 1 19 Los Patios, 
Los Gatos, CA 95032 (US). SHULER, Eric, Thomas 
[US/US]; 123 Palm Drive, Piedmont, CA 94610 (US). 
GRUNERT, Margaret, E. [US/US]; 983-7 Asilomar 
Terrace, Sunnyvale, CA 94086 (US). AVERY, Jason 
[US/US]; 1327 Milvia Street, Apt. 4, Berkeley, CA 94709 
(US). 

(74) Agents: JEWIK, Patrick et al.; Townsend and Townsend 
and Crew LLP, Two Embarcadero Center, 8th Floor, San 
Francisco, CA 94111 (US). 

(81) Designated States (national): AE, AG, AL, AM, AT (util- 
ity model), AT, AU, AZ, BA, BB, BG, BR, BY, BZ, CA, 
CH, CN, CO, CR, CU, CZ (utility model), CZ, DE (util- 
ity model), DE, DK (utility model), DK, DM, DZ, EC, EE 
(utility model), EE, ES, FI (utility model), FT, GB, GD, GE, 
GH, GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, 

[Continued on next page ] 



(54) Title: INTERACTIVE APPARATUS USING PRINT MEDIA 




(57) Abstract: A method for producing an output in response to an interaction with a print element on a sheet is disclosed. The 
method includes placing a sheet (27) comprising a print element on a surface of a base unit (12). A user can then mark on the sheet 
in the vicinity of the print element with a marking instrument (35) . An audio output (18) that corresponds to the print element is 
generated. 



WO 03/001475 ai MM IIIIHI II Nil Mill I II III 111 II llllllllllllllllllllllllllllllllllllll 



LC, LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, 
MW, MX, MZ, NO, NZ, OM, PH, PL, PT, RO, RU, SD, 
SE, SG, SI, SK (utility model), SK, SL, TJ, TM, TN, TR, 
TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZM, ZW. 

(84) Designated States (regional): ARIPO patent (GH, GM, 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZM, ZW), 
Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), 
European patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, 
GB, GR, IE, IT, LU, MC, NL, PT, SE, TR), OAPI patent 
(BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, ML, MR, 
NE, SN, TD, TG). 



Declaration under Rule 4.17: 

of inventorship (Rule 4.17 (iv)) for US only 

Published: 

— with international search report 

before the expiration of the time limit for amending the 
claims and to be republished in the event of receipt of 
amendments 

For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



WO 03/001475 



PCT/US02/19364 



INTERACTIVE APPARATUS USING PRINT MEDIA 

BACKGROUND OF THE INVENTION 

Various electronic book systems are known. One electronic book system is 
5 capable of recognizing print elements such as figures, letters, and codes that are printed on 
the sheets of a book. The book is mounted at a fixed location, and the user can arbitrarily 
designate a specific location to identify a particular print element on the sheet. A sensor 
system can detect the arbitrarily-designated location. A location detection circuit can, from a 
determination of an absolute location signal or a predetermined reference point, plus the 

10 arbitrarily-designated location point, determine the print element that the user selected. A 
ROM (read only memory) can be appropriately addressed by the sensor system to provide 
additional information corresponding print element. The ROM contains the location of the 
selected print element and related information for that location. The related information can 
be retrieved for subsequent processing, for example, by a computer-driven system that can 

15 provide audio and video images that are related to the selected print element. 

While such electronic book systems are useful, many improvements could be 
made. For example, such electronic book systems do not engage a user in any other activity 
than selecting a particular figure or letter. To help a user retain knowledge of a particular 
figure or letter, it would be desirable if the user could interact with the electronic book 
20 systems in other ways. Moreover, it would be desirable if the electronic book systems could 
have other features that could attract a user to use it. By doing so, a user would become more 
engaged with the book system. The electronic book system could thus be a more effective 
learning tool than such conventional systems. 

Embodiments of the invention address these and other problems. 

25 

SUMMARY OF THE INVENTION 

One embodiment of the invention is directed to a method of generating audio 
output, the method comprising: a) placing a sheet having a print element printed on the sheet 
on a surface of a base unit; b) marking on the sheet in the vicinity of the print element with a 
30 marking instrument; and c) generating an audio output that corresponds to the print element. 
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Another embodiment of the invention is directed to an interactive apparatus 
comprising: a) a base unit comprising i) a surface for receiving a sheet having a print element 
printed on the sheet, ii) an array of electrical elements under the surface, iii) a processor 
operatively coupled to the array of electrical elements, iv) a memory operatively coupled to 
5 the processor, and v) an audio output device operatively coupled to the processor; and b) a 
marking instrument that is adapted to mark on the sheet in the vicinity of the print element, 
wherein the audio output device is adapted to provide audio output that relates to the print 
element when a user makes a mark in the vicinity of the print element with the marking 
instrument. 

10 Another embodiment of the invention is directed to a system comprising: a) a 

server computer; b) an interactive apparatus comprising a base unit, wherein the base unit 
includes i) a surface for receiving a sheet having a print element printed on the sheet, ii) an 
array of electrical elements under the surface, iii) a processor operatively coupled to the array 
of electrical elements, iv) a memory operatively coupled to the processor, wherein the 

15 memory comprises an output that is related to the print element, and v) an output device 
operatively coupled to the processor, wherein the output device is adapted to generate the 
output after a user interacts with the print element while the sheet is on the surface; c) a client 
computer operatively coupled to the server computer; and d) a printer operatively coupled to 
the client computer. 

20 Another embodiment of the invention is directed to a method comprising: a) 

sending code for a sheet having a print element printed on the sheet from a server computer 
to a client computer at a user site; b) printing the sheet at the user site; c) placing the sheet on 
a base unit of an interactive apparatus, wherein the base unit includes a surface and an array 
of electrical elements under the surface; d) interacting with the print element while the sheet 

25 is on the base unit; and e) producing an output related to the print element in response to the 
interaction with the print element. 

These and other embodiments are described in further detail below. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 shows a plan view of an apparatus according to an embodiment of the 

30 invention. 
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FIG. 2 shows a plan view of a base unit of an interactive apparatus with a 
removable template that secures a sheet to the base unit. 

FIG. 3 shows a block diagram of an apparatus according to an embodiment of 

the invention. 

5 FIG. 4 shows an electrical schematic diagram of an array of electrical 

switches. 

FIG. 5 shows a top view of a lower portion of a group of switches in an array 

of switches. 

FIG. 6 shows a cross-sectional view of a portion of a switch as pressure is 
1 0 applied to the switch. 

FIG. 7 shows a top view of an interactive apparatus comprising a base unit 
with a hinge and two portions that move about the hinge. 

FIGS. 8(a) and 8(b) show cross-sectional views of marking instruments 
according to embodiments of the invention. 

15 FIG. 9 shows a block diagram of a system according to an embodiment of the 

invention. 

FIG. 10 shows a table of script that can be used in an embodiment of the 

invention. 

20 DETAILED DESCRIPTION 

In embodiments of the invention, a sheet such as a piece of paper can be 
placed on a base unit of the interactive apparatus. When the sheet of paper is on the base 
unit, the user may interact with a print element (e.g., drawings, words, letters) on the sheet. 
The interaction with the print element can be, for example, marking on the sheet in the 
25 vicinity of the print element. The vicinity of the print element can be on the print element or 
an appropriate distance around the print element. 

For instance, the user can color a print element such as a drawing on a sheet 
on the base unit with a marking instrument such as a crayon. In response to this, the base unit 
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can produce an output that relates to the drawing for the user. Illustratively, if the user is 
coloring the letter B on a sheet of paper, the output provided to the user may be synthesized 
speech such as "B, B says buh". In another example, the user can color an animated 
character that is printed on a sheet on the base unit with crayons or markers. In response to 
5 the coloring, the output provided to the user can be the sound of the animated character's 
voice. 

The interactive apparatus according to embodiments of the invention provide 
appropriate output in response to marking on a sheet. The user can mark the sheet in any 
suitable manner and with any desired colors. This enhances the user's creativity, 

1 0 self-expression, and imagination as the user can not only select a particular print element on 
the sheet, but can also mark on it. Accordingly, the user's creativity, self-expression, and 
imagination skills are used and developed. Also, when the user marks on a print element, the 
user is actively engaged with the print element. Additional information relating to the print 
element is provided in response to the marking. The additional information can reinforce or 

15 teach educational concepts conveyed by the print element. Because the user is engaged in an 
activity in addition to listening, the user is more likely to remember the user's interaction 
with the print elements on a sheet and is more likely to retain any concepts that are taught by 
the print elements and any additional audio information relating to the print elements. In 
addition, in embodiments of the invention, a child can be independently attracted to the 

20 interactive apparatus by an inviting activity (e.g., coloring, drawing) that the user is familiar 
with. In one example, children enjoy coloring sheets of paper with crayons and markers. 
The interactive apparatus can invite a child to color a print element on a sheet with crayons 
and markers of the child's choice. As the child colors the print element, output related to the 
print element is provided to the child in response to the coloring. When the child has finished 

25 coloring, the child has created a uniquely colored sheet and has learned about the print 
elements on the sheet in the coloring process. 

Any suitable sheet can be used with the interactive apparatus. The sheet can 
be made of plastic, paper, cardboard, etc., and can be colored or uncolored. In some 
embodiments, the sheet may be one of many sheets in a book. The book can have any 
30 suitable number of pages, and may include a binder such as a spiral or a ring binder. The 
sheet may also have any suitable size. For example, the sheet can have a standard size such 
as 5x7, 8.5x11, 8x14, or A4. 
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Any suitable number or types of print elements can be on the sheet. 
Exemplary print elements include drawings and portions thereof, words, phrases, portions of 
words, phonograms, shapes, pictures (e.g., photographs) and portions thereof, symbols, maps, 
letters, numbers, etc. 

5 In some embodiments, the print elements can be input symbols such as lines 

and boxes that can be designed to prompt a user for a response. Such input symbols are often 
found on forms such as test and questionnaire forms. Output (e.g., audio output) that is 
related to the correctness or incorrectness of the response, or audio output that helps the user 
provide a response (e.g., audio help information) can be provided to the user. For example, 

1 0 the print elements may be printed boxes that can be marked by the user to indicate a 

particular selection. For instance, a multiple choice form can have boxes that are associated 
with the letters a), b), and c) and three different answers. The user can select one of the 
answers by marking on one of the boxes with a check mark or an "x" mark. An audio 
response can be provided to the user in response to the marking. For example, the output 

15 "sorry, b) is incorrect" can be provided to the user after the user checks the box associated 
with the letter b), where the correct answer to the question posed by the interactive apparatus 
is the one associated with the letter a). Alternatively or additionally, the interactive apparatus 
may record the user's responses. In these embodiments, the user's responses may be 
recorded in the memory of the base unit of the interactive apparatus and on the sheet. A 

20 record of the user's responses can be recorded on the sheet and also electronically in the base 
unit of the interactive apparatus. 

The base unit of the interactive apparatus may have any suitable 
characteristics. In some embodiments, the base unit can comprise a major surface. The 
major surface may be uneven, but is preferably flat so that a user can write on it. The major 
25 surface can be smooth and washable so that markings such as crayon marks can be easily 
removed from it. 

An array of electrical elements is underneath the major surface of the base 
unit. In some embodiments, each electrical element is a pressure-sensitive switch that is 
activated by pressure. A suitable pressure-sensitive switch may comprise, for example, 
30 facing conductive regions that are separated by an air gap. Pressure applied to the conductive 
regions causes the facing conductive regions to contact each other closing the switch. 
Another type of electrical element may be a piezoelectric type element. The application of 
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pressure to a piezoelectric type element causes it to change resistance. In either case, 
pressure can be applied by, for example, a marking instrument such as a crayon that is above 
the electrical element. 

In other embodiments, the electrical elements can be antennas such as those 
5 described in U.S. Patent Nos. 5,877,458 or 5,686,705, and U.S. Patent Application Nos. 

09/574,599 and 60/200,725. All of these patents and patent applications are assigned to the 
same assignee as the present application and are incorporated herein by reference in their 
entirety for all purposes. The antennas can transmit signals that can be received by a stylus 
that is coupled to the base unit. The stylus acts as a receiving antenna. When the stylus is 

10 positioned over the major surface, the stylus receives a signal that is particular for the 

electrical element underneath the stylus. Then, the position of the stylus relative to the major 
surface can be determined. The stylus could be used to interact with print elements on a sheet 
that is secured to the base unit. The electrical elements may be transmitting antennas that 
regularly transmit signals that are received by the stylus, or may be receiving antennas that 

15 receive a signal from the stylus. In these embodiments, a user need not selectively activate 
the electrical elements. As explained below, the stylus can also be a marking instrument. 

In embodiments of the invention, some or all of the electrical elements in the 
array of electrical elements can be pre-assigned to retrieve and provide specific outputs for 
the user. Preferably, the electrical elements retrieve the pre-assigned outputs after they are 
20 activated. In addition, some of the electrical elements can be pre-assigned to indicate that a 
different sheet with different print elements is on the base unit when they are activated. For 
example, once an electrical element associated with a new sheet is activated, a processor can 
reprogram the interactive apparatus so that the electrical elements in the array are re-assigned 
to retrieve outputs associated with the print elements in the new sheet. 

25 The output provided to the user by the interactive apparatus may be visual 

output and/or audio output. Audio output is preferred as audio output can supplement and 
reinforce visual information such as letters, pictures, and numbers that may be on the sheets. 
The combination of both audio output and visual output reinforces concepts that are taught by 
the interactive apparatus. Exemplary output can include letters, stories, numbers, words, 

30 phrases, jokes, music, questions, answers, prompts, sound effects, facts, etc. Preferably, the 
audio output is provided in response to the user's act of marking on the sheet. 
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In some embodiments, the output can be a prompt that asks the user for a 
response. For example, an exemplary question that the interactive apparatus might ask the 
user may be "Can you find the letter P?" In response, the user can mark on the letter P on the 
sheet. After the user responds, output relating to the correctness or the incorrectness of the 
5 user's response can be provided to the user. For example, if the user's response is correct, a 
reward output (e.g., "that's right!") can be provided to the user. If the response is incorrect, 
the question can be repeated or the user can be informed that the response was incorrect. 

In other embodiments, the output provided by the interactive apparatus can 
include clues that help the user answer a question that was previously presented by the 

1 0 interactive apparatus. For example, a user may place a coloring sheet with the numbers 1 

through 10 on a base unit. An initial prompt by the base unit may be, for example, "Can you 
color the number 10?" If the user does not color the number "10" after a predetermined 
period of time, the interactive apparatus can prompt the user to obtain a clue by selecting 
another print element or pressing a button (e.g., in a removable template on the base unit or in 

15 the base unit). After the user selects the button or the print element, the apparatus can 
provide a clue such as "The number 10 comes after the number 9" to the user. Providing 
clues to the user can help the user learn about a given subject. 

As noted, in embodiments of the invention, the output provided to the user can 
relate to a particular print element with which the user is interacting. So, for example, the 
20 same output is not produced when the user colors the letter D and when the user colors the 
letter F. Rather, different outputs that specifically relate to the different print elements are 
presented to the user. For instance, sounds such as "dee" and "efff ' are provided to the user 
after the user colors on the letters D and F 5 respectively. 

The output can have some relationship to the information conveyed by the 
25 print element or the nature of the print element. For instance, the output can relate to the 
identity, form, color, shape, or quality of a print element on a sheet. For example, after 
coloring on a drawing of a frog on a sheet of paper on the base unit with a crayon, the phrase 
"Hi, I'm Leap!" can be retrieved from memory and a speaker in the base unit can sound the 
phrase to the user. Alternatively or additionally, the output can provide additional 
30 information about a particular print element. For example, a print element can be in the form 
of a state such as California. After the user interacts with the print element, information such 
as the population, the history, and/or the area of the state can be presented to the user. 
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In some embodiments, a removable template can be used to secure the sheet to 
the base unit. Preferably, the removable template can have bright colors and/or two or 
three-dimensional characters (e.g., animated figures). Features such as these can invite a 
child to interact with the interactive apparatus. The removable template may also be made of 
5 plastic and can be rigid or flexible. A rigid removable template is preferred as children can 
easily manipulate it. It is easier to align than a flexible template. In some embodiments, the 
removable template can be a substantially planar layer of molded plastic. Moreover, the 
removable template can have any suitable planar dimensions or thickness. For example, a 
typical removable template can have a thickness that is less than about 0.5 inches and planar 
10 dimensions greater than about 8.5x11 square inches. In other embodiments, a 

non-removable template can be used with the base unit. The non removable template can 
have the same or different features as the described removable template. 

The removable template can have a stationary portion and movable elements 
that move substantially independently of the stationary portion. The movable elements can 
15 be moved by a user and can interact with an electrical element underneath the movable 

element. For example, the user can apply pressure to the major surface of the base unit using 
a movable element. The applied pressure causes a pressure-sensitive switch underneath the 
movable element to activate. As a result of the activation, appropriate audio output can be 
presented to the user. 

20 Each movable element in the removable template may have a structure and an 

appearance that is distinct from the stationary portion. For example, in some embodiments, 
the movable elements may be buttons that are disposed in apertures in the stationary portion. 
The buttons are coupled to the walls of the apertures with flexible connectors that allow the 
buttons to move substantially independently of the stationary portion. In some embodiments, 

25 the movable elements can also be disposed in elongated apertures (i.e., with respect to the 
plane of the removable template) in the removable template. A movable element can move 
within an elongated aperture and also in a direction parallel to the orientation of the 
removable template. 

If a removable template is used to secure a sheet to the base unit, the 
30 removable template is preferably easily removable from the base unit. The user need not use 
any special tools or procedures to remove and separate the removable template from the base 
unit. Preferably, a removable template can rest on the major surface of the base unit without 
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the use of clips or adhesives. In this regard, different removable templates having different 
types, patterns, or sizes of movable elements can be interchangeably placed on the major 
surface of the base unit without difficulty. The interface presented to the user can be readily 
changed. As the removable templates are changed, the look, feel, and functionality of the 
5 interactive apparatus changes. In comparison to many conventional electronic toys, a child 
need not be presented with the same set of buttons and the same corresponding set of sounds 
each time the child uses the interactive apparatus. Consequently, children are less likely to 
become disinterested using embodiments of the invention than conventional electronic toys. 
Additional details regarding suitable removable templates that can be used in embodiments of 

10 the invention can be found in U.S. Patent Application No. , entitled 

"Interactive Apparatus with Templates", filed on the same day as this application (Attorney- 
Docket No, 020824-001300US). This patent application is assigned to the same assignee as 
the present application and is herein incorporated by reference in its entirety for all purposes. 

The interactive apparatus can be used for any purpose. For example, it can be 

15 used to entertain and/or educate the user. In some embodiments, the sheets can be designed 
so that the user can play games. For example, a sheet that is secured to the base unit may 
have a maze printed on it. The user can use a pencil or other marking instrument to trace a 
path through the maze. Pressure is applied to the major surface under the sheet using the 
pencil. Electrical elements underneath the pencil are activated and a processor in the base 

20 unit can electronically track the path of the pencil. Audio output (e.g., "wrong way!") can be 
provided to the user if the user strays from the correct path in the maze. In another example, 
the interactive apparatus can teach the user about specific print elements. For example, 
specific letters or numbers can be printed on a sheet. The output that is provided to the user 
may relate to the name or sound of the letters or numbers. Alternatively or additionally, the 

25 output can be an illustration of how a number or letter is used. In yet another example, 

embodiments of the invention can be used to teach the user kinesthetic skills. For example, 
the interactive apparatus can help a user learn how to write letters or numbers. For example, 
electrical elements underneath the major surface of the base unit can electronically track the 
path of the user's marking instrument. The interactive apparatus can determine if the user has 

30 successfully written a particular letter or number on a sheet. Appropriate feedback can be 
provided to the user depending upon whether the user has correctly written the particular 
letter or number on the sheet. After writing, the user can take the sheet and keep it or show it 
to another person (e.g., a teacher or parent). Accordingly, embodiments of the invention have 
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more functionality than simply providing declaratory instruction to a user. The user may, for 
example, practice handwriting skills using embodiments of the invention. 

In yet other embodiments, the sheets can be designed to specifically educate 
the user about a predetermined subject. For example, the interactive apparatus and the 
5 removable templates can be designed to teach a user about colors, letters, shapes, numbers, 
word pronunciation, phonics, current events, songs, general math, algebra, subtraction, 
multiplication, division, fractions, decimals, geometry, science, geography, history, spelling, 
grammar, the names and sounds of musical instruments, people, places, nature, music, sports, 
letters, numbers, counting, social studies, creative expression, languages such as English and 
10 Chinese, etc. The skills taught by the interactive apparatus can include recognition skills 

(e.g., number and letter recognition) and logic skills. Accordingly, in some embodiments, the 
interactive apparatus can be characterized as an educational toy. 

Although in some embodiments, the interactive apparatus has been described 
as an educational toy, in other embodiments, the interactive apparatus need not be a 

15 children's toy and could be used by adults. For example, in some embodiments, the different 
sheets that are used with the base unit can be designed to teach adults a second language. In 
another example, the sheet on the base unit may be a pre-printed form that is to be filled in by 
the user. The interactive apparatus can provide output that assists the user in filling out the 
form, provide appropriate feedback to the user's selections, and/or automatically and 

20 electronically record the user's selections on the form. 

The user of the interactive apparatus can be of any suitable age (e.g., an adult 
or child). For example, the user can be a child that is less than about 10 years old (e.g., a 
pre-school age child between about 3 to about 5 years old). In embodiments of the invention, 
the interactive apparatus engages children with talking characters, music, sounds, and fun 
25 learning activities. Important skills (e.g., preschool skills) can be taught to children while 
encouraging creativity, self-expression and imagination. 

An interactive apparatus embodiment can be described with reference to FIG. 
1. FIG. 1 shows a base unit 12 and a marking instrument 35 of an interactive apparatus 
according to an embodiment of the invention. The base unit 12 includes an edge region 17 
30 that surrounds a major surface 29 of the base unit 12. The edge region 17 and other parts of 
the base unit 17 can be made of molded plastic. The underside of the base unit 12 may have 
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a storage recess for storing one or more removable templates (not shown) and/or sheets when 
they are not in use. 

A handle 16 is attached to the edge region 17. The handle 16 may be 
retractable or detachable from the main body of the base unit. Using the handle 16, the user 
5 can easily transport the base unit and any removable templates or sheets that are used with the 
base unit (not shown). Accordingly, the interactive apparatus can be portable and can be 
carried by a young child. In this regard, the interactive apparatus can have planar dimensions 
that are less than about 3 ft. by about 3 ft. For example, the base unit 12 may have 
dimensions of approximately 13" x 18.75" x 3" (W x L x H). 

10 The edge region 17 of the base unit 12 can have any suitable features. For 

example, a covered or uncovered storage space 11 can be in the edge region 17. The storage 
space can store the user's personal items and marking instruments such as pencils or crayons. 
In addition, a speaker 18 is in the edge region 17 of the base unit. Pre-recorded or 
synthesized sounds can be provided to the user through the speaker 18. The sounds can be 

15 music, synthesized words, phrases, and word segments, sound effects, etc. A slot 23 (shown 
by invisible lines) for a transferable information storage medium (not shown) can also be 
included in the edge region 17. The slot 23 can be cooperatively structured to receive the 
transferable information storage medium. A spring-loaded door (not shown) can cover the 
slot 23 and can be pushed inward when, for example, a data cartridge is inserted into the slot 

20 23. 

A selecting device 20 is also in the edge region 17 to let the user select a 
particular mode of operation. As shown in FIG. 1, the selecting device 20 can be in the form 
a slideable knob. The user can use the selecting device 20 to turn the interactive apparatus on 
or off. When it is on, the user can further select from a learning mode, a game mode, and a 

25 music mode. In the learning mode, the primary output provided by the interactive apparatus 
can be output that teaches the user about a predetermined subject. For example, when the 
interactive apparatus is in the learning mode, the user can interact with a print element of a 
particular color or shape. In response, the interactive apparatus can state the shape and the 
color of the print element to the user. In the game mode, the user can play games or the 

30 interactive apparatus can quiz the user. For example, a sheet having the letters A through Z 
can be on the base unit. The interactive apparatus may prompt the user by saying "Can you 
color the letter D?" If the user colors the wrong letter, the interactive apparatus can tell the 
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user the name and sound of the letter (e.g., "B, B says buh."). If the user selects the right 
letter, the interactive apparatus can reward the user. For example, a congratulatory phrase 
such as "y° u did it!" or other rewarding sound effect could be provided to the user if the user 
correctly colors the letter D. Thus, the interactive apparatus can provide different types of 
5 output to the user depending upon whether the user marks on the right or wrong print 
element. In the music mode, different musical segments can be produced when the user 
marks on different print elements. For example, if a sheet with print elements is on the base 
unit, different musical segments can be presented to the user when the different print 
elements are colored. This entertains the user and keeps the user engaged with the interactive 
10 apparatus. 

The major surface 29 of the base unit 12 can have any suitable characteristics 
or size. The major surface 29 can be the upper surface of a layer of polyvinyl acetate or other 
suitable polymeric material. Preferably, the major surface 29 is made of a material that can 
be easily washed if, for example, crayon markings are made on the major surface. In some 
15 embodiments, the major surface 29 can have planar dimensions that are larger than a standard 
sheet of paper. For example, the major surface 29 can have planar dimensions that are more 
or less than about 8.5 x 14 inches. Such larger dimensions can accommodate a removable 
template that can secure a standard sheet of paper to the base unit 12. 

The edge region 17 in this and other embodiments can be cooperatively 
20 structured to receive a removable template or a sheet (not shown). In the illustrated example, 
the edge region 17 is raised with respect to the major surface of the base unit 12 and encircles 
the major surface. When, for example, a removable template or a sheet is placed over the 
major surface of the base unit 12, the edge region 17 guides the removable template or sheet 
so that it is correctly positioned over the major surface of the base unit 12. The print 
25 elements in the sheet are also correctly positioned over the electrical elements under the 
major surface of the base unit 12. Accordingly, in embodiments of the invention, the 
removable template or the sheet can be self-aligning with respect to the major surface of the 
base unit 12. This makes it easy for a user such as a child to place the removable template or 
the sheet on the surface of the base unit 12 and use the interactive apparatus. A user need not 
30 worry about aligning the removable template or the sheet so that the print elements perfectly 
correspond to the electrical elements underneath the major surface. 
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A marking instrument 35 may also be used with the base unit 12. The 
marking instrument 35 may be separated from the base unit 12 or coupled to the base unit 12. 
For example, the marking instrument 35 may be coupled to the base unit 12 with a retractable 
or a non-retractable wire. Examples of suitable marking instruments include crayons, 
5 markers, pencils, pens, etc. 

FIG. 2 shows an interactive apparatus comprising a removable template that 
secures a sheet to the base unit 12. The removable template 14 includes a hinge that joins a 
first portion 22 and a second portion (not shown) that is underneath the second portion. Both 
the first and the second portions may be in the form of frames. They can also be substantially 
10 planar. The second portion is underneath the first portion 22. The second portion may have a 
window (or a flexible membrane) of about the same size as the window in the first portion so 
that pressure can be applied through it and to the major surface under the removable template 
14. Together, the first portion and the second portion form a clamshell. 

The first portion 22 has a frame that defines a window. A user can lift the first 
15 portion 22 by grasping a raised portion 22(a) formed in the first portion 22. Lifting the first 
portion causes an actuator (not shown) at the bottom of the removable template 14 to apply 
pressure to an electrical element (e.g., a switch) underneath the removable template 14. A 
voice from the speaker 18 can then say, "Put in a new coloring page". At this point, some of 
the electrical elements can be locked out (i.e., non-responsive) until a sheet is loaded and the 
20 user presses an indicator (e.g., a green "Go" circle) that informs the base unit as to the 
particular sheet that is on it. The user can then place a sheet 27 between the first and the 
second portions. The user then closes the removable template by bringing the first portion 22 
next to the second portion. The sheet 27 can then be viewed through the window in the first 
portion 22. A voice from the speaker 18 may then say, "Let's color while we learn our 
25 ABCs". 

The user may depress the "Go" circle 24 (or other identifier) printed on the 
sheet 27 to inform the base unit that the sheet 27 is on it. A voice from the speaker may 
prompt the user with the phrase "touch the Go circle on your coloring page!" The user can 
then depress the "Go" circle with a crayon or finger. Depressing the "Go" circle causes an 
30 electrical element under the major surface to activate. Activating the electrical element under 
the "Go" circle informs the base unit that the particular sheet 27 shown in FIG. 2 is on the 
base unit 12. If the user presses the wrong section of the sheet 27 and activates an electrical 
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element designated for another sheet, the user can depress the reset button 15. After 
depressing this button, the interactive apparatus can reset itself and can prompt the user to 
"Press the green Go circle" to continue. 

The sheet 27 includes a number of print elements. The print elements on the 
5 sheet 27 include a large letter P, the word PIE, and a drawing of a frog eating a piece of pie. 
The user can color any of these print elements with crayons or markers. As the user colors a 
print element, pressure is applied to the sheet 27 and to the major surface that is underneath 
the sheet 27. The applied pressure activates one or more electrical elements (not shown) 
underneath the print element being colored. After the one or more electrical elements are 
10 activated, audio corresponding to the print element being colored by the user can be provided 
to the user through the speaker 18. For example, as the user is coloring the word "PIE", a 
synthesized voice coming from the speaker can say, "pie, pie is spelled P-I-E". In another 
example, if the user colors the picture of the frog eating the piece of pie, the audio provided 
to the user may be the sound of the character shown in the drawing eating a piece of pie. 

15 An exemplary script table that can be associated with the sheet 27 is shown in 

FIG. 10. As shown in FIG. 10, different audio outputs can be provided in response to the 
user's selection of different print elements. In FIG. 10, "dt" means "down touch" onto a print 
element and "drag" means that the user is dragging an instrument across a print element, 
"sfx" stands for "sound effects". 

20 After the user is finished interacting with the sheet 27, the sheet 27 can be 

replaced with a sheet with different print elements. For example, the sheet 27 may be 
replaced with a sheet that shows a large letter "A" and a picture of an ant. The user may 
inform the base unit, or the base unit may be automatically informed, that a different sheet is 
on the base unit. A processor in the base unit can then assign the electrical elements so that 

25 outputs that are specific to the print elements on the new sheet are produced. For example, 
the user can color the picture of an ant and a synthesized voice coming from the speaker can 
say, "Ant is spelled A-N-T". 

FIG. 3 shows a block diagram of some of the components in an interactive 
apparatus according to an embodiment of the invention. Each of the components shown in 
30 FIG. 3 can be housed in a base unit. Some of the components (e.g., the transferable 
information storage medium 80) can be physically separated from the base unit. 
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The interactive apparatus includes a processor 86. The processor 86 may be 
one or more microprocessors that facilitate the operation of the interactive apparatus. In 
embodiments of the invention, the one or more microprocessors may include an ASIC 
(application specific integrated circuit). 

The processor 86 may be coupled to any suitable electronic component. In 
FIG. 3, the processor 86 is coupled to a memory 90, an input device 92, and an array of 
electrical elements 88. A speech synthesizer 82 and an audio output device 84 provide 
synthesized speech or other audio output for the user. A transferable information storage 
medium 80 is removably coupled to the processor 80. In some embodiments, the transferable 
information storage medium 80 is insertable into a slot in the base unit. Data can be retrieved 
from the base unit or provided to the base unit through the transferable information storage 
medium. 

The speech synthesizer 82 can be a speech synthesizer chip that is separate 
from a microprocessor that facilitates the interaction between the various components in the 
interactive apparatus. Alternatively, the speech synthesizer 82 can be incorporated onto a 
computer chip along with logic circuitry that facilitates the interaction between the various 
components in the interactive apparatus. 

The audio output device 84 provides audio output for the user. Examples of 
audio output devices include speakers and earphone jacks. Other output devices such as 
visual output devices (not shown) can be used in addition, or as an alternative, to the audio 
output device 84. Examples of visual output devices include VGA (video graphic array) 
displays, LCDs (liquid crystal displays) and LEDs (light emitting diodes). 

The memory 90 may be any suitable temporary or permanent information 
storage device. The memory 90 may include optical, magnetic, or electronic storage media 
such as optical or magnetic disks, tapes, sticks, and the like. Any suitable combination of 
information storage devices can collectively make up the memory 90 of the interactive 
apparatus. In some embodiments, the memory 90 can include one or more RAM (random 
access memory), ROM (read only memory) and EPROM (erasable programmable memory) 
chips. For example, the memory 90 can include a ROM chip that has a storage capacity of 
about 2 megabytes or more. 
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The memory 90 can store any suitable information. For example, the memory 
90 can store electronic maps for the removable templates and code for the sheets that are used 
with the base unit. Each electronic map can associate a particular electrical element in the 
array of electrical elements with a particular output. The particular output can relate to a 
5 particular print element on a printed sheet. The memory 90 may store code for the user's 
name or code to retrieve the user's name from, e.g., a transferable information storage 
medium. The apparatus can greet the user with a first name so that the user's experience is 
personalized. Code for the outputs can also be stored in the memory 90. 

In some embodiments, information such as questions, words, pictures, musical 
10 sounds, applications, etc. may be stored in libraries of code, libraries of resources, and 

libraries of applications. These libraries can be present in the memory 90. Information such 
as this may be stored once in the memory 90 and may be accessed according to a set of 
instructions provided to the processor 86. Sets of instructions for manipulating data, rather 
than the data files themselves, can be quickly transferred to and from the interactive 
15 apparatus. Large data files such as sound and image files need not be stored in and 

repeatedly transferred to and from the base unit. Small instruction files may be transferred, 
thus resulting in faster data transfer and lower memory capacity requirements. Music data, 
for example, may be quickly transferred to and from the interactive apparatus according to a 
MIDI (Musical Instrument Digital Interface) or MIDI-like protocol. For example, in some 
20 embodiments, actual audio data need not be transferred to and from the base unit, just the 
instructions to play the audio data. This allows the same audio sequences stored in the base 
unit to be used multiple times with no additional audio storage required. 

Instructions to play audio data may include general audio sequences (GASs). 
GASs are very compact instructions for specifying and playing audio resources. A single 
25 GAS command sequence can specify a complicated sequence of audio. Exemplary GAS 
commands may include the following: 

GAS commands for hGAS HelloThere 
PlayhCA_Hello 
Wait 0.1 seconds 
30 PlayhCAJThere 

In this example, actual audio data for the words "Hello" and "There" need not be transferred 
to and from the base unit. These phrases can be stored once in the memory of the base unit 
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and can then be retrieved and played according to instructions that are transferred to and from 
the base unit. Instruction files are generally smaller in size than actual, audio files so that they 
can be transferred to and from the base unit more quickly than actual audio files. Thus, the 
interactive apparatus according to embodiments of the invention can be readily updated. 

5 The general audio sequences (GAS) can call various audio resources that may 

be stored as CA, SYN and RAW audio resources. Although any suitable labels may be used, 
SYN may be a short form for the word "synthesized" and may be used in conjunction with an 
on-chip synthesizer used for musical instruments, horn honk, animal sounds, etc. "SYN" 
files may be derived from musical instrumental digital interface (MIDI) type files. "RAW" 
10 may be raw, or uncompressed, audio sound files. They can be used when sounds of very high 
quality are desired. "CA" may stand for "compressed audio" files. 

Each CA audio resource is a compressed binary file representing a word, 
phrase, or sound. Each SYN audio resource may be a set of commands representing 
instruments and notes. In some embodiments, the actual instrument sound files are not stored 
15 with the SYN audio resource, just the commands. This allows musical sets of commands to 
specify complex musical pieces with very high quality in very small space. The instrument 
sounds and notes are stored in a RAW audio resource file which may be an uncompressed 
binary file representing a sound. It may be used when high quality audio output is desired. 

One or more input devices 92 can be included in the interactive apparatus. 
20 Examples of input devices include styluses, knobs, buttons, and levers. These may be in the 
edge region of the base unit. For example, the previously described selecting device can be 
an input device. Other input devices such as volume controls can also be in the base unit. 

Any suitable transferable information storage medium 80 may be used in the 
interactive apparatus. For instance, the transferable information storage medium 80 may be a 
25 cartridge, a disk, a tape, or a memory stick. Data cartridges are especially desirable since 
they can be easily manipulated and handled by children. In some embodiments, the 
transferable information storage medium 80 comprises a flash memory cartridge. 

The transferable information storage medium 80 can be used to provide new 
computer code to the interactive apparatus. Suitable computer code can include code that 
30 updates the interactive apparatus so that a new sheet can be used with the interactive 

apparatus. In addition, the code can also update the interactive apparatus so that other sheets 
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can be used with the interactive apparatus. Thus, by using the transferable information 
storage medium 80, the interactive apparatus can be reprogrammed so that new sheets with 
different print elements printed on them can be used with the interactive apparatus. It can 
also be used to transfer audio or graphics data as well as instructions to play audio or graphics 
5 data to the interactive apparatus. 

The transferable information storage medium 80 can also be used to retrieve 
computer code from the interactive apparatus so that it can be used elsewhere. For example, 
the user's performance information can be recorded in a user log file stored in the 
transferable information storage medium 80. The user's performance information can be 

10 logged to the user log file and the performance information can be uploaded to a server 

computer. Using the performance information and optionally a personal profile, the server 
computer can select content for the user. This performance information can be recorded in 
the user log in the transferable information storage medium 80 and can be uploaded to a 
server computer. The interactive apparatus can thus adapt to the user's learning ability over 

15 time so that new and different challenges are presented to the user. Adaptive learning 
processes and systems are described in greater detail in U.S. Patent Application No. 
09/632,424 filed August 4, 2000, which is assigned to the same assignee as the present 
invention and which is herein incorporated by reference in its entirety for all purposes. 

In some embodiments, a transferable information storage medium is not 
20 needed to transfer computer code to and from the base unit. For example, the base unit can 
have or be coupled to a transceiver (e.g., an RF transceiver) that can receive or send computer 
code to a computer that is separated from the base unit. In another example, the base unit can 
have a data port such as a USB (universal serial bus) port that allows a user to transfer data to 
and from the base unit through a physical connection. The physical connection may be a data 
25 cable coupled to a computer. 

The base unit of the interactive apparatus can be powered using batteries or a 
power cord. Preferably, the electronics in the base unit 12 are powered using batteries. In 
addition, the base unit 12 may automatically shut off when the interactive apparatus has not 
been used for a predetermined period of time (e.g., 1 or 3 minutes or more). By doing so, 
30 battery power can be conserved if the user forgets to turn off the interactive apparatus. 
Children are particularly likely to forget to turn off electronic toys after they are finished 
using them. 
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As noted, the array of electrical elements 88 can be located underneath the 
major surface of the base unit. The electrical elements in the array can be arranged in a 
regular or an irregular manner. For example, the electrical elements may be arranged in a 
plurality of rows and columns in the array. Each electrical element can correspond to a 
particular row and a particular column in the array of electrical elements 88. An exemplary 
schematic electrical diagram of an array of electrical switches is shown in FIG. 4. In the 
diagram, there are 8 rows and 14 columns for a total of 1 12 switches that are labeled 1 
through 112. Each switch can correspond to a particular row and a particular column. 

In an array of electrical elements, a portion of the electrical elements in the 
array may correspond to the print elements in one sheet, while other electrical elements in the 
array may correspond to print elements in a different sheet. For example, referring to FIG. 4, 
print elements in a first sheet may be designed to activate switches 47, 60, and 69 and provide 
outputs that are related to print elements on a first sheet. A second sheet can be designed to 
activate switches 71, 74, and 80 and provide outputs that are related to print elements on the 
second sheet. By designating certain switches within an array to be used with certain sheets, 
the electrical elements in the array of electrical elements need not be reassigned to retrieve 
different output for each sheet that is used with the base unit. 

In some embodiments, one or more electrical elements in the array of 
electrical elements 88 can be assigned to identify the particular sheet with one or more 
distinct print elements being used by the user. A print element can be provided in each of the 
sheets to activate a certain electrical element. The electrical element can be pre-assigned to 
inform the base unit that a different sheet is on the base unit. After being activated, certain 
electrical elements in the array of electrical elements 88 can be re-assigned to retrieve outputs 
that correspond to the new sheet that is on the base unit. 

Illustratively, again referring to FIG. 4, a first sheet (not shown) is placed over 
the array of switches. A "Go" button (or other indicator) in the upper right hand comer of the 
first sheet can be depressed to activate the switch 14. A synthesized voice from a speaker in 
the base unit can prompt the user to press the "Go" button. By activating the switch 14, the 
processor determines that the first sheet is on the base unit. Switches 85, 87, and 89 can be 
assigned to retrieve outputs that are specific to the print elements on the first sheet. After 
interacting with the print elements on the first sheet, a second sheet can be placed over the 
array of switches. The second sheet can have a "Go" button at the lower right hand corner. 
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The "Go" button in the second sheet can be disposed over switch 112. Other print elements 
in the second sheet can be disposed over switches 85, 87, and 89. After depressing the "Go" 
button on the second sheet, the switch 112 is activated and the processor determines that 
second sheet is on the base unit. The processor subsequently re-assigns switches 85, 87, and 
89 so that when they are activated, outputs that relate to the print elements on the second 
sheet are retrieved instead of outputs that relate to the print elements on the first sheet. 

As noted, the electrical elements can be pressure-sensitive switches. In some 
embodiments, each switch may have an upper conductive region and a lower conductive 
region that are not in contact with each other in the absence of applied pressure. The upper 
conductive region may be in a specific row while the lower conductive region may in a 
specific column (or vice versa). When pressure is applied to the upper conductive region, the 
upper conductive region contacts the lower conductive region closing a circuit and activating 
the switch. The processor determines the row and the column of the depressed switch. It can 
then retrieve output that was assigned to the activated switch from memory. This output can 
then be presented to the user. 

FIG. 5 shows a plan view of a portion of an array of switches according to an 
embodiment of the invention. Lower conductive regions 42(a), 42(b) correspond to columns 
in an array of switches. The lower conductive regions 42(a), 42(b) can be, for example, a 
conductive ink that is printed on an insulating substrate. The columns of lower conductive 
regions 42(a), 42(b) are electrically isolated from each other by insulating regions 46. A 
number of insulating spacers 44 are on the lower conductive regions 42(a), 42(b). 

The insulating spacers 44 space the lower conductive regions 42(a), 42(b) of 
the switches from their corresponding upper conductive regions (not shown). The upper 
conductive regions are separated from the lower conductive regions 42(a), 42(b) by an air 
gap. In the absence of applied pressure, the air gap separates the upper and lower conductive 
regions so that the switch is open. 

When pressure is applied to the switch, an upper conductive region contacts 
the lower conductive region of the switch thus forming a closed circuit. Referring to FIG. 6, 
pressure can be applied to a major surface 29 of the base unit. The user can apply pressure to 
the major surface 29 by using, for example, a finger or a marking instrument. 
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The major surface 29 can be on an upper insulating layer 52 that has an upper 
conductive region 54 printed on it. The upper insulating layer 52 can be made of, for 
example, Mylar™ or any other suitable flexible insulating material. Insulating spacers 44 
keep the upper conductive region 54 and a lower conductive region 42 on a lower insulating 
layer 60 from contacting each other. However, when pressure is applied to the major surface 
29, the upper insulating layer 52 and the upper conductive region 54 deform. The upper 
conductive region 54 and the lower conductive region 42 then contact each other, thus 
activating the switch. The conductive material making up the conductive regions may be 
solid or patterned. Patterning the conductive material saves cost, as less conductive material 
is needed to produce the interactive apparatus. 

The maximum lateral spacing of adjacent insulating spacers 44 that separate 
the upper and lower conductive regions of a switch may be adjusted so that a user's palm 
does not activate the switch while permitting a user's finger or a marking instrument to 
activate switch. For example, in some embodiments, the maximum lateral spacing between 
adjacent insulating spacers 44 can be about 0.125 inches to about 0.5 inches (or less). By 
doing so, the user can rest the user's palm on the major surface of the base unit, without 
inadvertently activating the switches underneath the major surface. 

The locations at which contact is made between upper conductive region 54 
and lower conductive region 42 may correspond to a pattern marked or noted on an 
interactive activity sheet or template that is placed on major surface 29. In the interactive 
apparatus embodiments, a set of paired conductive regions 42 and 54, may be included in a 
pattern that denotes the outlines of a block letter "A". There may be, for example, 10 pairs 
of contact points underlying the outlines of a block letter "A" on an activity sheet. When a 
crayon is used to color in the letter "A" on the activity sheet, the set of 10 contact pairs are 
repeatedly, sequentially, activated, and the interactive apparatus responds with an audio 
signal. For example, the audio may say, "That's the letter A. It says "aeh" or "aaeee" as in 
apple or April" The interactive apparatus may continue to play other audio as long as the 
letter "A" is being colored in. For example, it may go on to play the song "April Showers" or 
say the poem, "An apple a day keeps the doctor away". If a child is coloring in the block 
letter "A", the coloring activity may not be completely continuous. The child may stop for a 
few seconds, holding the crayon off the sheet. When the child begins to start coloring again, 
it is not necessarily desirable to have the audio begin from the beginning, particularly if the 
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child's pause in activity was only for a couple of seconds. Because traditional "debounce" 
techniques are symmetric, use of a traditional debounce technique would delay the onset of 
signal acknowledgement when the child started to color as well as delaying the restart of 
signal after a pause. (A traditional debounce technique would count to a pre-set number of 
5 repeat contacts before acknowledging a signal as bonefide, e.g. not spurious, and wait for the 
same pre-set number of contacts in a row to be missed before acknowledging that the signal 
was terminated.) It would not be desirable to have a delay in the onset of audio when the 
child began to draw, so known debounce techniques were not useful to cure the problem of 
the audio starting over each time the child paused for a few seconds. Consequently the 

10 present interactive apparatus was designed to benefit from an inventive asymmetric debounce 
technique. The initiation of contact within a set of contacts is termed a "down transition" and 
the cessation of contact within a set of contacts is termed an "up transition". In the inventive 
asymmetric debounce technique, for a given sampling rate for electrical signal activity, the 
down transition requires only a single time sample of the contact set to show an electrical 

15 signal, to acknowledge a signal as bonefide (or f, good"). In contrast, the up transition 

requires 39 time samples in a row to be missed, that is to occur with out an electrical signal 
being detected, to acknowledge a cessation in activity. If an electrical signal is detected any 
time before the 39 th time sample, the audio resumes as if there had been no interruption at all 
(the values 1 and 39 were selected for purposes of illustration; the asymmetric debounce 

20 technique generally only desirably uses values that are very different and therefore any 
suitable values may be used). Thus, unwarranted jumps to the beginning of the audio are 
avoided. Accordingly, in embodiments of the invention, if the user does not interact with a 
sheet within a predetermined of time, the audio may start over, and may not continue from 
the prior audio played to the user. 

25 In the embodiments described above, the base unit is generally a single, 

integral body. However, in other embodiments, the base unit may have a hinge so that 
portions of the base unit fold on each other. FIG. 7 shows a base unit 800 with a hinge 810. 
A book 815 with print elements 817 may be on the base unit 800. A speaker 802 is in the 
base unit 800 and a cartridge 804 is in a slot (not shown) at a side of the base unit 800. An 

30 indentation 806 outside the border provides a location to store a stylus 808 when the platform 
is brought to a closed configuration. An optional stylus holder 812 may be included to hold 
the stylus 808. The stylus 808 has the ability to mark on the book 815. In this example, the 
book 815 includes a number of print elements 817 in the form of the numbers 1, 2, and 3. 
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The user can mark on these print elements with the stylus 808 and audio output respectively 
relating to the numbers 1, 2, and 3 can be presented to the user. Other details regarding 
hinge-type base units can be found in U.S. Patent Application No. 09/777,262, filed February 
5, 2001. This patent application is herein incorporated by reference for all purposes and is 
assigned to the same assignee as the present application. 

As noted above, in some embodiments, the base unit may comprise an array of 
electrical elements that are antennas. In these embodiments, the stylus that is used with the 
base unit may comprise a receiving antenna and may be coupled to the base unit with a wire. 
The stylus may be a marking instrument that holds, for example, a pencil or a crayon. When 
the user makes a mark in the vicinity of a specific print element with the stylus, the antenna 
in the stylus can receive a signal specific to antenna that is directly under the antenna in the 
stylus. Two examples of such styluses are shown in FIGS. 8(a) and 8(b). In FIGS. 8(a) and 
8(b), the wires to the styluses are not shown. FIG. 8(a) shows a stylus including a housing 
78. The stylus housing 78 houses an antenna and an ink reservoir 72. Ink in the ink reservoir 
72 may feed a tip region 76 of the stylus. FIG. 8(b) shows a stylus with a stylus housing 78 
and an antenna 74. In this embodiment, the stylus can hold, for example, a marking element 
80 such as a crayon or a pencil lead. Unlike the embodiment shown in FIG. 8(a), a separate, 
preformed, marking element 80 can be inserted and secured into the stylus housing 78. 

INTERACTIVE SYSTEMS 

In some embodiments of the invention, computer code that can be used in the 
interactive apparatus can be sent (e.g., downloaded) to the user from a remote server 
computer. The code may be for different outputs for different sheets and/or for different print 
elements on the different sheets (or instructions to play the outputs). Code for producing 
sheets with different print elements can also be sent to the user. The code for the sheets may 
be in the form of standard graphics files such as bitmap files or JPEG files. Once the code for 
the sheets is downloaded to the user site, the sheets can be printed at the user's site using a 
printer. After the sheets are printed, they can then be used with the interactive apparatus. 
Moreover, electronic maps for the downloaded sheets can also be downloaded to the user site. 
In some embodiments, any or all of the code for the sheets, the electronic maps, and the code 
for the outputs associated with the print elements on the sheets can be included in a content 
package that can be sent to from the remote server computer to the user's site. 
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A system according to an embodiment of the invention can be described with 
reference to FIG. 9. The system includes a host site 109 and a user site 120. The user site 
120 may be, for example, a home, school, or workplace. The host site 109 and the user site 
120 are coupled through a communication medium 107. Many user sites may be present. 
However, only one user site is shown for simplicity of illustration. 

The host site 109 may include a server computer 105 operated by a host 
company. In this example, the host site 109 has a server computer 105 that can access a 
content database 103 and a personal profile database 101. 

The server computer 105 is typically a powerful computer or cluster of 
computers. For example, the server computer can be a large mainframe, a minicomputer 
cluster, or a group of servers functioning as a unit. In one example, the"server computer may 
be a database server coupled to a Web server. The server computer can behave as a single 
computer that services the requests of one or more client computers. For example, one or 
more client computers 109 at different user sites can communicate with the server computer 
through the communication medium 107 and optionally through one or more Internet Service 
Providers (ISPs) (not shown). The client computers typically have less memory and less 
computing power than the server computer 105. The server computer 105 can be remotely 
located with respect to the client computer 109. 

The server computer 105 can have computer code for performing any suitable 
function. For example, the server computer 105 can have code for updating a user profile in 
a user profile database, retrieving data from any suitable database, and sending data to any 
person. # Computer code may also be provided for selecting content for a content package, 
assembling the content package, and sending the content package to the user. The content 
package may include code for audio outputs, code for graphic outputs, code for instructions 
to play audio or graphics output, code for sheets with print elements, code for personalized 
information, code for electronic maps associating the print elements with specific outputs, 
etc. 

The server computer 105 and the client computer 109 communicate through a 
communication medium such as the Internet, and more particularly the World Wide Web 
(WWW). Typical communication protocols that can be used by the server computer 105, the 
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client computer may include HTTP (hypertext transfer protocol), TCP/IP (terminal 
communication protocol /Internet protocol), or other protocol. 

The server computer 105 may have data representing a Web site. The Web 
site may have any suitable number of Web pages and is preferably entertaining to users under 
5 18 years of age. On the Web site, the user (or other person such as the user's parents) can, 
for example, create a user profile, set operational preferences, play games, and select content 
for a content package. 

The personal profile database 101 may include the user's personal preferences 
and information about the user. The user profile information may include basic information 
10 about the user. Such basic information includes the user's age, hobbies, address, phone 
number, e-mail address, school, relatives, teachers, parents, phone number, etc. The user 
profile information may also include information regarding the user's preferences. User 
preferences may include preferences such as favorite subjects, relatives, teachers, favorite 
songs, favorite sports, favorite foods, and favorite cartoons. 

15 Information in the personal profile database 101 can be used to identify 

information that can be downloaded to the user site 120 and then transferred to the interactive 
apparatus 113. By doing so, the user's experience can be personalized. For example, the 
user's profile may indicate that the user likes sports. Code for sheets with math problems or 
sentences that relate to sports can be selected for the user and can be downloaded by the user 

20 for use in the interactive apparatus. 

The content database 103 may include any suitable content. The content may 
include information that can be used in the interactive apparatus. Examples of such 
information include problems (e.g., questions, puzzles, riddles, quizzes), as well as songs, 
games, hints, music, facts, greetings, and messages (e.g., help, encouragement, reminders, or 
25 error messages). The content database 103 can also include code for electronic maps for 

different sheets. In addition, computer code for sheets with print elements can be present in 
the content database 103. The code can be downloaded to the user site 120 and the sheets can 
then be printed at the user site 120 using a printer. 

The content in the content database 103 may also be in any suitable form. For 
30 example, the content may be in the form of compressed or uncompressed data. The content 
may be in the form of sound files such as MP3 files, WAV files, and MIDI (Musical 
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Instrument Digital Interface) files. The content may also be in the form of graphic files such 
as video files, image files (e.g., bitmaps), and text files. 

At the user's site 120, a client computer 109 can be operatively coupled to a 
linker device 111 and a printer 115. The printer 115 and the linker device 111 may be 
5 coupled to the client computer 109 through a wireless or a wired link (e.g., a cable). The 
interactive apparatus 113 can receive computer code from the server computer 105 through 
the client computer 109 via the linker device 111. 

The client computer 109 and the printer 115 may have any suitable 
characteristics. For example, the client computer 109 can be an ordinary personal computer 
10 running a Windows™ based operating system. It may have a suitable Internet browser such 
as Microsoft Explorer™. The printer 115 may be any suitable printing'apparatus. Suitable 
examples of printers include thermal printers, laser printers, ink-jet printers, dot matrix 
printers, etc. 

The linker device 111 may be any suitable device capable of transferring data 
15 between the client computer 109 and the interactive apparatus 113. The linker device 111 
could be, for example, a wireless transceiver such as a wireless RF (radio frequency) 
transceiver or an infrared transceiver. In preferred embodiments, however, the linker device 
111 is capable of reading information from, and writing information to, the transferable 
information storage medium. In a typical embodiment, the transferable information storage 
20 medium may be a cartridge that can be inserted into both the linker device 111 and the base 
unit of the interactive apparatus 113. Using the linker device 111, data can be transferred to 
the interactive apparatus directly or indirectly. 

In an illustrative use of the system shown in FIG. 9, a user can use the client 
computer 109 to contact the server computer 105 through the communication medium 107. 

25 While visiting a Web site on the server computer 105, the user may select specific sheets with 
specific print elements for downloading. Alternatively or additionally, the server computer 
105 can then retrieve specific content for the user from the content database 103 using the 
user's personal profile in the personal profile database 101. All content can then be collected 
together to form of a content package. The content package can include, for example, code 

30 for new sheets that can be printed using a standard printer, electronic maps associating the 
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print elements in the sheets with particular locations on a base unit and particular outputs, and 
code for producing specific outputs in the interactive apparatus. 

The client computer 109 can receive the content package. The user may then 
insert a data cartridge into the linker device 111 and code for the electronic maps and the 
5 code for producing the specific outputs can be loaded onto the data cartridge using the linker 
device 111. The user can then remove the data cartridge from the linker device 111 and can 
plug the transferable information storage medium into the base unit of the interactive 
apparatus 113. In the base unit, the newly transferred computer code can update the 
functionality of the interactive apparatus so that it can be used with the downloaded sheets. 

10 Before or after code is loaded into the base unit of the interactive apparatus, 

the user can preview the sheets on a display device (not shown) such as~a VGA display 
device coupled to the client computer 109. After previewing the sheets using the display 
device, the sheets can be printed out using the printer 115, which is also coupled to the client 
computer 109. These sheets can be used with the updated interactive apparatus 113 in the 

1 5 manner previously described. 

The system shown in FIG. 9 has a number of advantages. For example, by 
using the system shown in FIG. 9, a number of different sheets with different print elements 
can be downloaded to the user's site. Computer code (e.g., electronic maps) used to 
reprogram the interactive apparatus for new sheets can also be downloaded. Accordingly, the 
20 user is not limited to using a particular set of sheets that might be available at, for example, a 
local store. The user can have ready access to new and different content so that the 
interactive apparatus is more interesting to the user. 

In some embodiments, the interactive apparatus can be used with other types 
of devices to form a set of devices that can be used, for example, to educate a child as the 

25 child grows. Each device (e.g., an educational toy) can be designed to educate a child in a 
certain age range. For example, the interactive apparatus according to embodiments of the 
invention can be designed to help educate children between the ages of 3 to 6 years old. 
Another device, such as an embodiment that is shown and described in U.S. Patent No. 
6,142,784 (which is assigned to the same assignee as the present invention and which is 

30 herein incorporated by reference in its entirety), can be used to educate children between the 
ages of 7 to 12 years of age. Other devices can be used to educate children from the ages 12 
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to 16, 1 to 3, etc. This set of devices can be used together as part of a personalized tutorial 
service so that a child has different types of devices that can complement each other and help 
a child learn as the child grows. 

The terms and expressions which have been employed herein are used as 
terms of description and not of limitation, and there is no intention in the use of such terms 
and expressions of excluding equivalents of the features shown and described, or portions 
thereof, it being recognized that various modifications are possible within the scope of the 
invention claimed. Moreover, any one or more features of any embodiment of the invention 
may be combined with any one or more other features of any other embodiment of the 
invention, without departing from the scope of the invention. 
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1. A method of generating audio output, the method comprising: 

a) placing a sheet having a print element printed on the sheet on a surface 

of abase unit; 

b) marking on the sheet in the vicinity of the print element with a marking 
instrument; and 

c) generating an audio output that corresponds to the print element. 

2. The method of claim 1 wherein the print element comprises a letter and 
wherein the audio output comprises information relating to the letter, 

3. The method of claim 1 wherein b) comprises marking on the print 
element with the marking instrument. 

4. The method of claim 1 wherein the sheet is a sheet of paper and the 
marking instrument is a crayon. 

5. The method of claim 1 wherein the print element comprises a letter and 
wherein the audio output comprises the sound of the letter, a description of the letter, or a 
word that begins with the letter. 

6. The method of claim 1 wherein c) is performed before b). ' 

7. The method of claim 1 wherein the base unit comprises a surface and 
an array of electrical elements underneath the surface of the base unit. 

8. The method of claim 1 wherein the base unit comprises a surface and 
an array of pressure-sensitive elements underneath the surface of the base unit, and wherein 
the method further comprises: 

applying pressure to a pressure-sensitive element within the array of 
pressure-sensitive elements. 

9. The method of claim 1 wherein the base unit includes two portions 
coupled together with a hinge. 
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1 10. An interactive apparatus comprising: 

2 a) a base unit comprising 

3 i) a surface for receiving a sheet having a print element printed on the 

4 sheet, 

5 ii) an array of electrical elements under the surface, 

6 iii) a processor operatively coupled to the array of electrical elements, 

7 iv) a memory operatively coupled to the processor, and 

8 v) an audio output device operatively coupled to the processor; and 

9 b) a marking instrument that is adapted to mark on the sheet in the vicinity of 

1 0 the print element, 

1 1 wherein the audio output device is adapted to provide audio output that relates 

12 to the print element when a user makes a mark in the vicinity of the print element with the 

1 3 marking instrument. 

1 11. The interactive apparatus of claim 1 0 wherein the marking instrument 

2 is separated from the base unit. 

1 12. The interactive apparatus of claim 10 wherein the marking instrument 

2 includes a receiving antenna. 

1 13. The interactive apparatus of claim 1 0 wherein the print element 

2 comprises a letter. 

1 14. The interactive apparatus of claim 10 wherein the base unit comprises 

2 two portions coupled together with a hinge. 

1 15. The interactive apparatus of claim 10 further comprising a removable 

2 template that secures the sheet to the base unit. 

1 16. The interactive apparatus of claim 10 wherein the interactive system 

2 further comprises a transferable information storage medium, and wherein the base unit 

3 further comprises a slot for the transferable information storage medium. 

1 17. The interactive apparatus of claim 10 wherein the marking instrument 

2 is physically coupled to the base unit. 
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1 18. The interactive apparatus of claim 10 wherein the marking instrument 

2 is a crayon. 

1 19. The interactive apparatus of claim 10 wherein the audio output device 

2 is a speaker. 

1 20. The interactive apparatus of claim 10 wherein the array of electrical 

2 elements is an array of pressure-sensitive switches. 

1 21 . The interactive apparatus of claim 10 wherein the interactive apparatus 

2 uses an asymmetric debounce technique to determine if the user is interacting with the sheet 

3 within a predetermined period of time. 

1 22. A system comprising: 

2 a) a server computer; 

3 b) an interactive apparatus comprising a base unit, wherein the base unit 

4 includes 

5 i) a surface for receiving a sheet having a print element printed on the 

6 sheet, 

7 ii) an array of electrical elements under the surface, 

8 iii) a processor operatively coupled to the array of electrical 

9 elements, 

10 iv) a memory operatively coupled to the processor, wherein the 

1 1 memory comprises an output that is related to the print element, and 

12 v) an output device operatively coupled to the processor, wherein the 

13 output device is adapted to generate the output after a user interacts 

14 with the print element while the sheet is on the surface; 

15 c) a client computer operatively coupled to the server computer; and 

16 d) a printer operatively coupled to the client computer. 

1 23. The system of claim 22 further comprising a communication medium 

2 between the server computer and the client computer, wherein the communication medium is 

3 the Internet. 
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1 24. The system of claim 22 wherein the array of electrical elements is an 

2 array of pressure-sensitive switches. 

1 25. The system of claim 22 further comprising the sheet wherein the sheet 

2 is a coloring page. 

1 26. The system of claim 22 wherein the interactive apparatus further 

2 comprises a marking instrument. 

1 27. The system of claim 22 wherein the interactive apparatus further 

2 comprises a marking instrument coupled to the base unit. 

1 28. A method comprising: 

2 a) sending code for a sheet having a print element printed on the sheet from a 

3 server computer to a client computer at a user site; 

4 b) printing the sheet at the user site; 

5 c) placing the sheet on a base unit of an interactive apparatus, wherein the base 

6 unit includes a surface and an array of electrical elements under the surface; 

7 d) interacting with the print element while the sheet is on the base unit; and 

8 e) producing an output related to the print element in response to the 

9 interaction with the print element. 

1 29. The method of claim 28 wherein interacting with the print element 

2 comprises applying pressure to the print element. 

1 30. The method of claim 28 wherein the array of electrical elements 

2 comprises an array of pressure sensitive electrical elements. 

1 31. The method of claim 28 wherein the array of electrical elements 

2 comprises an array of antennas. 

1 32. The method of claim 28 wherein interacting with the print element 

2 includes marking on the sheet in the vicinity of the print element. 

1 33. The method of claim 28 wherein the print element is a letter, number, 

2 or symbol. 
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34. The method of claim 28 wherein the output is an audio output. 

35. The method of claim 28 wherein interacting comprises marking on 

print element. 



33 



WO 03/001475 



PCT/US02/19364 



1 /11 




SUBSTITUTE SHEET (RULE 26) 



WO 03/001475 



2/11 



PCT/US02/19364 




SUBSTITUTE SHEET (RULE 26) 



WO 03/001475 



PCT/US02/19364 



3/11 






_l 


CO 


u. < 


O O 


-z. 


>? * 


LLl 


C£ LU 


LEMI 


< — ' 
LU 


LU 



7 

CO 

oo 



CO 

CD 

LL_ 



SUBSTITUTE SHEET (RULE 26) 



WO 03/001475 



PCT/US02/19364 



4/11 





a jo 


3 jo s 

i "Us 




i 4 


» k!> 

« c 


SW112 




4,1 4 


I ^ 


5 Jo 8 




« q 


i 4 


1 






* 5 








s hi 

• ^ 


1 




4« 4 


• £ 


s hi 55 

% "L s 


4,i 4 


- 


■ ^ 


g 


^ O ^- OJ CO 

;8888 




FT 


s jo s 


2" 2 


1 

- lj 




§ 
1 




• 5 






s jo 

• 5 


i ■£ 


o 

i 




« j* 
« £ 


SS JO s 

* Ijl 






« £ 


1 


4« ^ 




so v -3- 




t 4 


1 4 


s 
1 




s 4 


S JO § 


i.i 4 


■ 5 


s hi 

» 5 


1 






8 JO u 






3 hi 

« 5 


S ! 
1 




« hi 

■ ^ 


S3 jo <? 




i 4 


« 5 


Cv] 

1 


0 % 




5 JO $ 

2 


% i C 

2 2 


R jA 

§ c 

11 


is jA 


1 


4« ^ 


« 5 


8 JO ? 

* Iji 


4* 4 


Pi ji 


8 hi 

• 5 


1 


4« 4 




a jo ? 


4* 4 


s 4 


8 jA 

i 5 


a> 

CO | 1 

i-cvim^incDscoa 

88888888E 


— 

"I ROWO 
R0W1 
R0W2 
R0W3 


ill! 

*: ce a cc 



|i-CMcoTfin<oh»oo I o 

15 1 ^° SUBSTITUTE SHEET (RULE 26) 



WO 03/001475 



PCT/US02/19364 



5/11 




42(a) 42(b) 46 



L 



r 



P 



FIG. 5 



48 




FIG. 6 



SUBSTITUTE SHEET (RULE 26) 



WO 03/001475 



PCT/US02/19364 



6/11 




SUBSTITUTE SHEET (RULE 26) 



WO 03/001475 



PCT/US02/19364 



7/11 



1 o 



CO O 
h- CO 



^1- 



CO 




oo 
CD 

LI- 



SUBSTITUTE SHEET (RULE 26) 



WO 03/001475 



PCT/US02/19364 



8/11 




O 

CNJ 



O 



CO 



Z 



LINKER 
DEVICE 


< ► 


INTERACTIVE 
APPARATUS 










CLIENT 
COMPUTER 




PRINTER 


► 



SUBSTITUTE SHEET (RULE 26) 



WO 03/001475 



PCT/US02/19364 



9/11 



Activity Mode - Page P 



Shape/Object 


Software cues 


Click type 


Activity mode" audio*Leap 
as narrator 




Underlining music plays 
while waiting for a Correct 
answer 




underlying music 
(BGCTAM1) 


Question #1 


///ere can p& rnree 
questions which can he 
nmnr&mtnpd $te*ctuf*ntiallv 

fjt <-*y/ hi i in icu oc^vyc/c^/ many, 

so that the child doesn't get 
the same seauence of 
questions each time. 
Default to music alter 
question is asked 


Question #1 of 3 w/o 
any coloring 


Can you find the lette 

n p"9 


Question #1 Hints 


pjier i si o seconos ot 
inactivity 




1 a\ WoVo lrinlri nn for tho 
\<A) Vvcic iuui\niy iui u it? 

letter "P" 




After 2nd 5 seconds of 
inactivity 




1a) Can you find the 
letter "P M 




After 3rd 5 seconds of 
inactivity, go to sfeep 




1 c) Thanks for learning with 
LeapFrog! Bye-Bye ! 


Correct Answer: 

Big Shape P (Big Letter) 


Kiii music 


dt or drag 


a1. Correct answer sfx 






dt or drag 


a2. Hooray! You found 
the letter "P" 


Correct Answer: Letter "P" 
with line under it 


Ki/i music 


dt or drag 


a1 . Correct answer sfx 






dt or drag 


a2. Way to go! You found 
the letter "P" 


Correct Answer: word pie 
Inside bubble 


Kiii music 


dt or drag 


a1 . Correct answer sfx 






dt or drag 


a2, Great Job! You found 
the letter "P" in the word pie. 




Please apply audio to all 
questions: After a correct 
answer response 


dt or drag 


Lets piay again! 


Incorrect Answer: 
Hint #1 


After 1st incorrect answer 


dt or drag 


a1. [Shape phrase]. Were 
looking for the letter "P". 


Hint #2 


After 2st incorrect answer 


dt or drag 


a2. [Shape phrase]. We're 
looking for the letter M P". 
The letter P js Dig! 


Hint #3 


After 3rd incorrect answer. 
Question #2 will kick in 
when finished. 


dt or drag 


a3. [Shape phrase]. Let's try 
another one. 


Shape Phrase: 
Leap 




dt or drag 


1 . I'm Leap! 


Tad 




dt or drag 


1. I'm Tad! 


The Pie 




dt or drag 


1. That's a piece of pie! 


Edison 




dt or drag 


1. I'm Edison! 


Letter "I" 




dt or drag 


1. That's the letter "I"! 


Letter "E" 




dt or drag 


1. That's the letter "E"! 


The Pan 




dt or drag 


null 


Question #2 


Question #2 will kick in after 
question #7 is finished. 
Default to music after 
question is asked 


dt or drag - Question 
#2 of 3 


Can you find the 
piece of pie? 



FIG. 10 

SUBSTITUTE SHEET (RULE 26) 



WO 03/001475 



PCT/US02/19364 



10/11 



Activity Mode - Page P 



Question #2 Hints 


After 1st 5 seconds 
or inaciiviiy 




2a) We're looking for the 

ni e» r*<s of nio 




After 2nd 5 seconds 
of inactivitv 




2b) Can you find the 

niApp nf nip? 




After 3rd 5 seconds 
of inactivity, go to sleep 




2c) Thanks for learning with 
Leap Frog \ Bye-Bye! 


Correct Answer: 
The piece of pie 


Kill music 


dt or drag 


a1. Correct answer sfx 




Kill music 


dt or drag 


az. nooray! you Touna 
the piece of pie! 


Correct Answer: word pie 
inside bubble 


Kill music 


at or crag 


a1 . Correct answer sfx 




Kill music 


a i or arag 


a2. Hooray! You found 1 
the word pie! 




Please apply audio to ail 
questions: After a correct 
answer response 


dt or drag 


Let's play again! 


Incorrect Answer: . 
Hint #1 


After 1st incorrect answer 


dt or drag 


a1. [Shape phrase], We're 
looking for the piece of pie! 


Hint 


After 2nd incorrect answer 


dt or drag 


a2. [Shape phrase]. We're 
looking for the piece of pie! 


Hint #3 


After 3rd incorrect answer. 
Kjuesiion ??o win kick in 
when finished. 


dt or drag 


a3. [Shape phrase]. Let's try 
another one. 


Shape Phrase: 
Leap 




dt or drag 


1 . I'm Leap! 


Tad 




dt or drag 


1 . I'm Tad! 


Edison 




dt or drag 


1. I'm Edison! 


Big Letter "P" / Letter "P" 




dt or drag 


1. That's the letter "P"! 


Letter "1" 




dt or drag 


1 . That's the letter "I"! 


Letter "E" 




dt or drag 


1 . That's the letter "E"! 


The Pan 




dt or drag 


null 


Question #3 


Question #3 will kick in after 
question #2 is finished. 

question is asked 


dt or drag 
Question #3 of 3 


Can you find Tad? 


Question #3 Hints 


/-l/ic?/ rot J ocl/U/iuo 

of inactivity 




3a^ We're lookino for Tad 




After 2nd 5 seconds 

Lit It ICtKsLIVIty 




3b) Can you find Tad? 




After 3rd 5 seconds 
of inactivity go to sleep 




3c) Thanks for learning with 
LeapFrog! Bye-Bye! ! 


Correct Answer: Tad 


Kill music 


dt or drag 


a1. Correct answer sfx 






dt or drag 


a2. Great Job! 
You found Tad! 




Please apply audio to ail 
questions: After a correct 
answer response 


dt or drag 


Let's play again! 


Incorrect Answer: 
Hint #1 


After 1st incorrect answer 


dt or drag 


a1. [Shape phrase]. We're 
looking for Tad! 


Hint #2 


After 2st incorrect answer 


dt or drag 


a2. [Shape phrase]. We're 
looking for Tad! 


Hint #3 


After 3rd incorrect answer. 
Question §1 will kick in 
when finished. 


dt or drag 


a3. [Shape phrase]. Let's try 
another one. i 



FIG. 10 

SUBSTITUTE SHEET (RULE 26) 



WO 03/001475 



PCT/US02/19364 



11 /11 



Activity Mode - Page P 



Shape Phrase: 
Leap 




dt or drag 


1. Pm Leap! 


The piece of pie 




dt or drag 


1. That's a piece of pie! 


Word inside bubble 




dt or drag 


1. That's the word pie! 


Edison 




dt or drag 


1. I'm Edison! 


Big Letter "P" / Letter "P" 




dt or drag 


1. That's the letter "P"! 


Letter "l" 




dt or drag 


1. That's the letter T! 


Letter "E M 




dt or drag 


1. That's the letter "E"! 


The Pan 




dt or drag 


null 



FIG.10 



SUBSTITUTE SHEET (RULE 26) 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US02/ 19364 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC(7) : G09G 5/00, 5/08, 5/39; G08C 21/00; G06T 1/60; 
US CL rPlease See Extra Sheet. 
According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

U.S. : 345/156,157,173,174,179,530,531,535,537; 178/18.01,18.02,18.03,18.05,18.06,18.07, 19.01,19.03,19.04 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields 
searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



X 



US 5,604,517 A (FILO) 18 February 1997; figures 1-9; summary; 
column 3, line 60- col. 10, line 48. 

US 5,767,457 A (GERPHEIDE et al) 16 June 1998, summary; 
figs. 1-7; col. 6, line 13- col. 11, line 5. 



1-21 



22-35 



j j Further documents are listed in the continuation of Box C. | [ See patent family annex. 



* Special categories of cited documents: 

"A 11 document defining the genera) state of the art which is not 

considered to be of particular relevance 

"E" earlier document published on or after the international filing date 

"L" document which may throw doubts on priority claim(s) or which is 

cited to establish the publication date of another citation or other 
special reason (as specified) 

"O" document referring to an oral disclosure, use, exhibition or other 

means 

"P" document published prior to the international filing date but later 

than the priority date claimed 



later document published after the international filing date or priority 
date and not in conflict with the application but cited to understand 
the principle or theory underlying the invention 

document of particular relevance; the claimed invention cannot be 
considered novel or cannot be considered to involve an inventive step 
when the document is taken alone 

document of particular relevance; the claimed invention cannot be 
considered to involve an inventive step when the document is 
combined with one or more other such documents, such combination 
being obvious to a person skilled in the art 

document member of the same patent family 



Date of the actual completion of the international search 



IS AUGUST 2002 



Date of mailing of the international search report 

,1 NOV 2BK ' 




Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington, D.C. 20231 
Facsimile No. (703) 305-3230 



IJAY SHANKAR 
Telephone No. (703) 305-4763 



Form PCT/rSA/210 (second sheet) (July 1998)* 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US02/19364 



Box I Observations where certain claims were found unsearchable (Continuation of item 1 of first sheet) 



This international report has not been established in respect of certain claims under Article 17(2)(a) for the following reasons: 
1. I I Claims Nos.: 

because they relate to subject matter not required to be searched by this Authority, namely: 



□ 



Claims Nos.: 

because they relate to parts of the international application that do not comply with the prescribed requirements to 
such an extent that no meaningful international search can be carried out, specifically: 



| | Claims Nos.: 

because they are dependent claims and are not drafted in accordance with the second and third sentences of Rule 6.4(a). 



Box II Observations where unity of invention is lacking (Continuation of item 2 of first sheet) 



This International Searching Authority found multiple inventions in this international application, as follows: 
Please See Extra Sheet. 



1. I x| As all required additional search fees were timely paid by the applicant, this international search report covers all 

searchable claims. 

2. As all searchable claims could be searched without effort justifying an additional fee, this Authority did not invite payment 
of any additional fee. 

3. | | As only some of the required additional search Fees were timely paid by the applicant, this international search report 

covers only those claims for which fees were paid, specifically claims Nos.: 



4. | j No required additional search fees were timely paid by the applicant. Consequently, this international search report is 
restricted to the invention first mentioned in the claims; it is covered by claims Nos.: 



Remark on Protest | | The additional search Fees were accompanied by the applicant's protest. 

[~~] No protest accompanied the payment of additional search fees. 
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A. CLASSIFICATION OF SUBJECT MATTER: 

US CL : 

345/156,157,173,174,179,530,531,535,537; 178/18.01,18.02,18.03,18.05,18.06,18.07, 19.01,19.03,19.04 

BOX II. OBSERVATIONS WHERE UNITY OF INVENTION WAS LACKING 
This ISA found multiple inventions as follows: 

1. Group I, claim(s) 1-21, drawn to the interactive apparatus for generating audio output, classified in class 345, 
subclass 173. 

2. Group II, claim(s) 22-35, drawn to a system comprising a server computer and memory, classified in class 345, 
subclass 507. 

3. The inventions are distinct, each from the other because of the following reasons: Inventions I— II are disclosed as 
different combinations which are not connected in design, operation or effect These combinations are independent if it 
can be shown that (l) they are not disclosed as capable of use together, (2) they have different modes of operation, (3) 
they have different functions, or (4) they have different effects. (MPEP 806.04, MPEP 808.01). In the instant case the 
combinations are based on vastly different functions ranging from the interactive apparatus for generating audio 
output, and a system comprising a server computer and memory. 

4. Because these inventions are distinct for the reasons given above and have acquired a separate status in the art as 
shown by their different classification, restriction for examination purposes as indicated is proper. 

5. Because these inventions are distinct for the reasons given above and the search required for Group I is not 
required for Group II, restriction for examination purposes as indicated is proper. 
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